Mean opinion score (MOS) is a numerical indication of the perceived quality of received media after compression and/or transmission. The MOS is expressed as a single number in the range 1 to 5, where 1 is lowest perceived audio quality, and 5 is the highest perceived audio quality measurement. Historically, mean opinion scores were generated by averaging the results of a set of standard, subjective tests in which a number of listeners listed to test sentences read aloud by both male and female speakers over the communications medium being tested. This listeners rated the audio quality of the test sentence according to a score ranging from 1 (bad quality or very annoying impairment) to 5 (excellent quality or imperceptible impairment.) A score for the communications medium was calculated as the arithmetic mean of all the individual opinion scores—thus the name “mean opinion score”. Eventually, algorithms were developed that calculated a mean opinion score without the need for human listeners. These algorithms were based on a mathematical analysis of observed or modeled degradation of audio waveforms or test tones, such as samples of human speech.
For example, ITU-T Recommendation G.107, hereinafter referred to as “ITU-T G.107”, defines what is known as the E-model, which is a computational model for use in transmission planning. This model includes an algorithm to calculate a transmission rating factor, R, which is a numerical value between 0 (extremely low quality) and 100 (extremely high quality.) The algorithm considers factors such as signal to noise ratio, absolute delay, simultaneous and delayed impairments, packet-loss probability, and quantizing distortion. Annex B of ITU-T G.107 also defines a function to map R values between 0 and 100 into MOS values between 1 and 4.5. (The highest R value, 100, is less than the highest MOS value, 5.)
However, the E-model defined in ITU-T G.107 is, as the name suggests, an algorithm for modeling how satisfied or unsatisfied a user will be with a particular communication system, including particular codecs, etc., once that system is built. The E-model gives an estimation of voice transmission quality, which can be transformed to give estimates of customer opinion, but in several places, including Annex C of ITU-T G.107, it is made clear that such estimates are only made for transmission planning purposes and is not suitable for actual (e.g., not modeled) customer opinion prediction.
ITU-T Recommendation P.862, hereinafter referred to as “ITU-T P.862”, defines a method that is suitable for end-to-end speech quality assessment. This method is called the perceptual evaluation of speech quality, or PESQ. PESQ compares an original signal to a degraded version of the same signal and calculates a numerical score between −0.5 and 4.5, which roughly corresponds to a MOS score of between 1.0 and 4.5. PESQ attempts to compensate for delays, frequency distortion, and gain loss introduced into the degraded signal before comparing it to the original signal. The final PESQ score is a linear combination of the average disturbance value and the average asymmetrical disturbance value.
However, PESQ performs its comparison on a decoded data stream; as data packets are received their contents must be extracted and used as input into the PESQ algorithm. An algorithm that operates on the contents of data packets is herein referred to as an “intrusive” algorithm, because the algorithm must intrude into the contents of the packets. Because the packet contents must be extracted and reassembled in order, intrusive algorithms are computationally more expensive than “non-intrusive” algorithms, which do not need to have access to the packet contents. ITU-T P.862 acknowledges the intrusive nature of the PESQ algorithm, and Table 2 of ITU-T P.862 specifically lists “In-service non-intrusive measurement devices” as an application for which PESQ is known to be inaccurate, e.g., that PESQ is not intended for use for non-intrusive measurement devices.
In light of the disadvantages described above, there exists a need for non-intrusive calculation of customer opinion prediction based on actual network performance. Accordingly, there exists a need for methods, systems, and computer readable media for non-intrusive mean opinion score (MOS) estimation based on packet loss pattern.